Method and system for displaying and navigating an optimal multi-dimensional building model

ABSTRACT

A method and system is provided for automatic generation and navigation of optimal views of facades of multi-dimensional building models based on where and how the original images were captured. The system and method allows for navigation and visualization of facades of individual or multiple building models in a multi-dimensional building model visualization system.

CROSS-REFERENCE TO RELATED APPLICATIONS/INCORPORATION BY REFERENCE

The present U.S. Utility patent application claims priority pursuant to35 U.S.C. §120 as a continuation-in-part of U.S. Utility applicationSer. No. 15/255,952, entitled “METHOD AND SYSTEM FOR DISPLAYING ANDNAVIGATING BUILDING FACADES IN A THREE-DIMENSIONAL MAPPING SYSTEM,”filed Sep. 2, 2016, which is a continuation of U.S. Utility applicationSer. No. 14/339,992, entitled “METHOD AND SYSTEM FOR DISPLAYING ANDNAVIGATING BUILDING FACADES IN A THREE-DIMENSIONAL MAPPING SYSTEM,”filed Jul. 24, 2014, now U.S. Pat. No. 9,437,044, which claims prioritypursuant to 35 U.S.C. §119(e) to U.S. Provisional Application No.61/858,241, entitled “METHOD AND SYSTEM FOR DISPLAYING AND NAVIGATINGBUILDING FACADES IN A THREE-DIMENSIONAL MAPPING SYSTEM,” filed Jul. 25,2013, all of which are hereby incorporated herein by reference in theirentirety and made part of the present U.S. Utility patent applicationfor all purposes.

The present application is related to the following:

1. U.S. Utility application Ser. No. 13/624,816, entitled,“THREE-DIMENSIONAL MAP SYSTEM,” filed on Sep. 21, 2012, now U.S. Pat.No. 8,878,865 and

2. U.S. Utility application Ser. Nos. 12/265,656, 12/265,656, entitled“METHOD AND SYSTEM FOR GEOMETRY EXTRACTION, 3D VISUALIZATION ANDANALYSIS USING ARBITRARY OBLIQUE IMAGERY,” filed Nov. 5, 2008, now U.S.Pat. No. 8,422,825.

BACKGROUND Technical Field

The technology described herein relates generally to an interactivethree-dimensional (3D) map system and, in particular, to a system andmethod for visualizing 3D building models within an interactive 3Dbuilding model visualization system.

Location-based and mobile technologies are often considered the centerof the technology revolution of this century. Essential to these areways to best present location-based information to electronic devices,particularly mobile devices. The technology used to represent thisinformation has traditionally been based on a two-dimensional (2D) map.

Efforts have been made to generate a three-dimensional (3D) map of urbancities via aerial imagery or specialized camera-equipped vehicles. 2Dmaps presenting street level photographs as well as 3D maps displaying3D building models have emerged as alternatives to classical 2D maps.However, these 2D or 3D maps have limited texture resolution andgeometry quality, and do not allow users to display the highest quality,optimal views of selected building models on a map. These efforts alsodo not integrate street-level imagery into a full 3D scene as theyrequire dual data interfaces between aerial-derived 3D datasets and adisorienting transition to static panoramic street-level imagery,typically collected using vehicles. Ways of resolving these problemshave been sought by those in the art, but prior developments have nottaught or suggested any viable solutions.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates one embodiment of a system for generating an optimalview of a 3D building model in accordance with the present disclosure;

FIG. 2 illustrates a diagrammatic representation of a machine in theexample form of a computer system in accordance with the presentdisclosure;

FIGS. 3A and 3B illustrate example embodiments of building models from athree-dimensional building model visualization system in accordance withthe present disclosure;

FIG. 4 illustrates an example embodiment of calculating a cameraposition for a 3D building model in accordance with the presentdisclosure;

FIG. 5 illustrates a flowchart embodiment of a process for generatingoptimal views of selected building models in accordance with the presentdisclosure;

FIG. 6 illustrates a flowchart of an exemplary embodiment forcalculating a camera position in accordance with the present disclosure;

FIGS. 7A-7B collectively illustrate examples of optimal views for 3Dbuilding models in a 3D building model visualization system inaccordance with the present disclosure;

FIG. 8 illustrates a general schematic of a multiple view building modelin accordance with the present disclosure; and

FIG. 9 illustrates an embodiment for displaying and navigating a 3Dbuilding model visualization system in accordance with the presentdisclosure.

DETAILED DESCRIPTION

One or more embodiments of the technology described herein include amethod and system for visualizing and navigating 3D building models.Optimal views of the facades (sides) of 3D building models areautomatically generated for display. In another embodiment, the systemand method provides for navigation through a 3D map system using one ormore optimal views of facades.

FIG. 1 illustrates one embodiment of a system for generating an optimalview of a 3D building model in accordance with the present disclosure.In one embodiment, 3D building model visualization system 100 includesview processing system 102 and electronic device 104 coupled via anetwork channel 106. Network channel 106 is a system for communication.Network channel 106 includes, for example, an Ethernet or otherwire-based network or a wireless network or wireless adapter forcommunicating with a wireless network, such as a WI-FI network. In otherembodiments, network channel 106 includes any suitable network for anysuitable communication interface. As an example and not by way oflimitation, network channel 106 can include an ad hoc network, apersonal area network (PAN), a local area network (LAN), a wide areanetwork (WAN), a metropolitan area network (MAN), or one or moreportions of the Internet, cloud based systems or a combination of two ormore of these. One or more portions of one or more of these networks maybe wired or wireless. As another example, network channel 106 can be awireless PAN (WPAN) (such as, for example, a BLUETOOTH WPAN), a WI-FInetwork, a WI-MAX network, a 3G or 4G network, a cellular telephonenetwork (such as, for example, a Global System for Mobile Communications(GSM) network).

In one embodiment, network channel 106 uses standard communicationstechnologies and/or protocols. Thus, network channel 106 can includelinks using technologies such as Ethernet, 802.11, worldwideinteroperability for microwave access (WiMAX), 3G, 4G, LTE, CDMA,digital subscriber line (DSL), etc. Similarly, the networking protocolsused on network channel 106 can include multiprotocol label switching(MPLS), the transmission control protocol/Internet protocol (TCP/IP),the User Datagram Protocol (UDP), the hypertext transport protocol(HTTP), the simple mail transfer protocol (SMTP), and the file transferprotocol (FTP). In one embodiment, the data exchanged over networkchannel 106 is represented using technologies and/or formats includingthe hypertext markup language (HTML) and the extensible markup language(XML). In addition, all or some of links can be encrypted usingconventional encryption technologies such as secure sockets layer (SSL),transport layer security (TLS), and Internet Protocol security (IPsec).

Electronic device 104 is a device for selection and display of a 3Dbuilding model. For example, electronic device 104 is a computer with amonitor, a laptop, a touch screen display, a smartphone, tabletcomputer, an LED array, game console, a television set, a projectordisplay, a wearable heads-up display of some sort, or any combinationthereof. In one embodiment, electronic device 104 includes a computersystem, such as computer system 200 of FIG. 2, for processing 3D visualdata for display.

In one embodiment, view processing system 102 collects buildingselection information (e.g., address, geo-location, position or areaswithin a displayed map, etc.) corresponding to a selected 3D buildingmodel from electronic device 104. In some embodiments, electronic device104 directly uploads selection information to view processing system 102via the network channel 106, or indirectly uploads the information. Forexample, the information is uploaded to a computer or a server firstbefore being uploaded to view processing system 102. For anotherexample, the information is transferred from electronic device 104 to anetworked computer, and then the information is transferred to the viewprocessing system 102.

FIG. 2 illustrates a diagrammatic representation of a computer system inaccordance with the present disclosure. In one embodiment, computersystem 200 includes a machine and a set of instructions for causing themachine to perform any one or more of the methodologies or modulesdiscussed herein. Computer system 200 includes a processor, memory,non-volatile memory, and an interface device. Various common components(e.g., cache memory) are omitted for illustrative simplicity. Thecomputer system 200 is intended to illustrate a hardware device on whichany of the components depicted in the example of FIGS. 1-2 (and anyother components described in this specification) can be implemented.Computer system 200 can be of any applicable known or convenient type.The components of computer system 200 can be coupled together via a busor through some other known or convenient device.

This disclosure contemplates computer system 200 taking any suitablephysical form. As example and not by way of limitation, computer system200 may be an embedded computer system, a system-on-chip (SOC), asingle-board computer system (SBC) (such as, for example, acomputer-on-module (COM) or system-on-module (SOM)), a desktop computersystem, a laptop or notebook computer system, an interactive kiosk, amainframe, a mesh of computer systems, a mobile telephone, a personaldigital assistant (PDA), a server, or a combination of two or more ofthese. Where appropriate, computer system 200 may include one or morecomputer systems; be unitary or distributed; span multiple locations;span multiple machines; or reside in a cloud, which may include one ormore cloud components in one or more networks. Where appropriate, one ormore computer systems 200 may perform without substantial spatial ortemporal limitation one or more steps of one or more methods describedor illustrated herein. As an example and not by way of limitation, oneor more computer systems 200 may perform in real time or in batch modeone or more steps of one or more methods described or illustratedherein. One or more computer systems 200 may perform at different timesor at different locations one or more steps of one or more methodsdescribed or illustrated herein, where appropriate.

The processor may be, for example, a conventional microprocessor such asan Intel Pentium microprocessor or Motorola power PC microprocessor. Oneof skill in the relevant art will recognize that the terms“machine-readable (storage) medium” or “computer-readable (storage)medium” include any type of device that is accessible by the processor.

The memory is coupled to the processor by, for example, a bus. Thememory can include, by way of example but not limitation, random accessmemory (RAM), such as dynamic RAM (DRAM) and static RAM (SRAM). Thememory can be local, remote, or distributed.

The bus also couples the processor to the non-volatile memory and driveunit. The non-volatile memory is often a magnetic floppy or hard disk, amagnetic-optical disk, an optical disk, a read-only memory (ROM), suchas a CD-ROM, EPROM, or EEPROM, a magnetic or optical card, or anotherform of storage for large amounts of data. Some of this data is oftenwritten, by a direct memory access process, into memory during executionof software in computer system 200. The non-volatile storage can belocal, remote, or distributed. The non-volatile memory is optionalbecause systems can be created with all applicable data available inmemory. A typical computer system will usually include at least aprocessor, memory, and a device (e.g., a bus) coupling the memory to theprocessor.

Software is typically stored in the non-volatile memory and/or the driveunit. Indeed, for large programs, it may not even be possible to storethe entire program in the memory. Nevertheless, it should be understoodthat for software to run, if necessary, it is moved to a computerreadable location appropriate for processing, and for illustrativepurposes, that location is referred to as the memory in this paper. Evenwhen software is moved to the memory for execution, the processor willtypically make use of hardware registers to store values associated withthe software, and local cache that, ideally, serves to speed upexecution. As used herein, a software program is assumed to be stored atany known or convenient location (from non-volatile storage to hardwareregisters) when the software program is referred to as “implemented in acomputer-readable medium.” A processor is considered to be “configuredto execute a program” when at least one value associated with theprogram is stored in a register readable by the processor.

The bus also couples the processor to the network interface device. Theinterface can include one or more of a modem or network interface. Itwill be appreciated that a modem or network interface can be consideredto be part of computer system 200. The interface can include an analogmodem, ISDN modem, cable modem, token ring interface, satellitetransmission interface (e.g., “direct PC”), or other interfaces forcoupling a computer system to other computer systems. The interface caninclude one or more input and/or output devices. The I/O devices caninclude, by way of example but not limitation, a keyboard, a mouse orother pointing device, disk drives, printers, a scanner, and other inputand/or output devices, including a display device. The display devicecan include, by way of example but not limitation, a cathode ray tube(CRT), liquid crystal display (LCD), or some other applicable known orconvenient display device. For simplicity, it is assumed thatcontrollers of any devices not depicted reside in the interface.

In operation, computer system 200 can be controlled by operating systemsoftware that includes a file management system, such as a diskoperating system. One example of operating system software withassociated file management system software is the family of operatingsystems known as Windows® from Microsoft Corporation of Redmond, Wash.,and their associated file management systems. Another example ofoperating system software with its associated file management systemsoftware is the Linux™ operating system and its associated filemanagement system. The file management system is typically stored in thenon-volatile memory and/or drive unit and causes the processor toexecute the various acts required by the operating system to input andoutput data and to store data in the memory, including storing files onthe non-volatile memory and/or drive unit.

According to one or more embodiments of the technology described herein,an “optimal view” of a 3D building model façade is defined as apreferred view of a 3D building model (e.g., elevated front view) whichis generated by a 3D building model visualization system from a bird'seye view/perspective and which calculates and generates an unobstructedor a partially unobstructed view of a 3D building model or group ofconnected or close proximity (e.g., same city block) models. Thegenerated view is provided to a computer system display as a centeredimage including at least one façade (side) of the 3D building model. The3D building model refers to a textured 3D building model using knowntexturing techniques

In one embodiment, the generated optimal view of the 3D building modelincludes visible buffer regions around the centered view of the building(top, bottom, left, right). In one or more embodiments, these bufferregions are white space around a selected 3D building object. In otherembodiments, the buffer regions include elements of building modelsand/or other visible structures located in the vicinity of the selectedbuilding model(s) within the 3D map.

FIGS. 3A and 3B illustrate embodiments of 3D building models from the 3Dbuilding model visualization system in accordance with the presentdisclosure. FIG. 3A illustrates one embodiment of a building model where3D building model 300 is selected to be displayed on electronic device104 of FIG. 1. In one embodiment, the 3D building model is selected onthe electronic device using current global positioning system (GPS)information. In alternative embodiments, building model 300 is selectedby touching or clicking (e.g., using a mouse/stylus) on a location of amap on the display of electronic device 112 of FIG. 1 or typing in anaddress of the desired 3D building model.

In one embodiment, electronic device 104 displays a view of buildingmodel 300 in the form of a view of building model façade 304. First mainaxis 301 and second main axis 302 are calculated axes (e.g., following aprimary roof line, for example a lengthwise building peak or orthogonalto the other axis) which define the orientation of the building model.As will be described in greater detail hereafter, the disclosed 3Dbuilding model visualization system, in accordance with the presentdisclosure, automatically calculates and/or pre-stores previouslycalculated optimal views for each façade of the selected or stored 3Dbuilding models. In one or more embodiments, an optimal view forbuilding model façade 304 is calculated based on a defined look angle303 with respect to calculated main axis 301 and a defined field of view(e.g., defined up/down, left/right angles which define the extent ofwhat is viewed on a 3D map) for the selected building. The calculatedoptimal view is centered on the display screen. A buffer zone (e.g.,white space or elements of surrounding model structures) around thefaçade of the building model is typically included in the displayed viewfor centering purposes. A primary optimal view is the optimal view whichthe 3D building model visualization system ranks as the highest qualityoptimal view available for display. In one or more embodiments, theprimary optimal view is a view generated for the front façade of a 3Dbuilding model. A primary axis is a first main axis perpendicular to theside of the building model which is displayed in the calculated primaryoptimal view. In one embodiment, selection of a building model on a 3Dmap will result in the 3D building model visualization systemcalculating optimal views for all the sides of a building model anddisplaying only the primary optimal view of the building model. In otherembodiment, the 3D building model visualization system will provideinterface controls to select one or more of the calculated optimal viewsof the building model. In one embodiment, the building model façade 304is facing a main road or street in the 3D building model visualizationsystem and the primary optimal view (typically the front) is selected tobe one of the optimal views which face a main street or road path 305from the 3D building model visualization system.

FIG. 3B illustrates another embodiment of a building model from the 3Dbuilding model visualization system in accordance with the presentdisclosure. In one embodiment, building model 300 has a primary optimalview selected on side 307 of the 3D building model which is not facing aroad. The optimal view is calculated and displayed based on a definedlook angle 306 and second main axis 302. In one embodiment, machinelearning capable algorithms are used to identify and determine aselected 3D building model's architectural features (e.g., windows, baywindows, doors, columns, railings, etc.) and rank the various facades ofa selected 3D building model based on expected architectural featurestypically located on primary facades (e.g., centered front door,portico, porch, high number of windows, etc.). In additionalembodiments, known neural networks or statistics such as archetypalanalysis on buildings or imagery are used to determine a pattern as towhat feature combinations and layouts generally constitute the primaryoptimal view of a building model within the 3D building modelvisualization system.

In one or more embodiments, optimal views and navigation within the 3Dbuilding model visualization system are achieved through pre-calculated,stored building model views and interbuilding navigation routes (e.g.roads, intersections, etc.). As the 3D map system is navigated, storedcalculated optimal views of the building models are displayed. The 3Dmapping system routes and views are then navigated in anobject-oriented, spatially intuitive (e.g., up, down, left, right)manner. The system simplifies industry common 3D navigational approachestypically referred to as “six degrees of freedom” methods that requirefine motor adjustments that are not experienced in navigating a 3Dmapping system. In other words, the navigator can navigate in 3D bysequencing through optimal views of one or more building models ratherthan trying to navigate in 3 dimensions using, for example, a 3D mouseor joystick.

FIG. 4 illustrates an example embodiment of calculating a cameraposition for a 3D building model in accordance with the presentdisclosure. In one embodiment, camera position 401 is arranged forgenerating optimal views for one or multiple building models 400 in a 3Dbuilding model visualization system. In one embodiment, projectionpyramid 401 having forward direction vector 402, right vector 403 and upvector 404 projecting from the camera point are used to calculate acamera position. Additional details for calculating a camera positionare discussed in FIG. 6.

FIG. 5 illustrates an embodiment of a flowchart of the process forgenerating optimal views of selected 3D building models in accordancewith the present disclosure. Process 500 is initiated when one or more3D building models are retrieved from the 3D building modelvisualization system (step 501) based on a selected building location.As previously stated, the location is selected using current globalpositioning system (GPS) information (e.g., a user is standing in frontof a building of interest), by touching or clicking on a location of amap, or typing in an address of the desired 3D building model. In oneembodiment, the 3D building models for the selected location areretrieved from the 3D map system.

In a next step, an optimal look angle (downward pitch angle) is defined(step 502) for the selected 3D building model. In one embodiment anoptimal look angle, as depicted in FIGS. 3A-3B (303/306) is set as adefault downward pitch angle relative to the calculated axis by the 3Dbuilding model visualization system (e.g., 10-50 degrees). For example,in one embodiment an optimal look angle is selected at or proximate to20 degrees. In alternative embodiments, the optimal look angle isdefined to limit obstructions from the view of the 3D building modelfacades. For example, if a tall structure is in the line of site of theoptimal view, the look angle can be raised to look over the obstruction.

In one embodiment, defining a look angle is determined largely by whereand how the images are taken during capture. Contextual (e.g.,telemetric) information about this capture is written into an EXIF forthe image. EXIF—exchangeable image file format is a standard thatspecifies formats for images, sound, and ancillary tags used by digitalcameras (including smartphones, drones, etc.). The contextualinformation is included with each image via an EXIF tag or separatelyvia a metadata file. The image metadata (contextual information)includes, but is not limited to:

GPS Latitude Ref

GPS Longitude Ref

GPS Altitude

GPS Dilution of Precision

GPS Img (image) Direction Ref (reference)

GPS Img (image) Direction

Additional metadata for gravity, camera orientation and image capturesequence number. For example, {“device_orientation”:3,“gravity”:

{“x”:-0.07310568541288376,“y”:-0.8332936763763428,“z”:0.5479755997657776},

“hover_cube_face”:3,“version”:“3.1.43”}—Image naming convention. Ex.84463244-6c58-4422-8389-d6991748ee05_3.jpg

The sequence number effectively describes and associates each image witha virtual location. For example, the user is directed to start imagecapture at a front door, go counter-clockwise to front right corner,right side, back left corner, back side, back right corner, side, frontleft corner in the image capture sequence and a name is associated withthe images with this sequence. If any of the camera sequence needs to beretaken (e.g., loss of image, bad imagery, etc.), the system preservesthe sequence number and increments the retry number, for example“84463244-6c58-4422-8389-d6991748ee05_3_1.jpg”, implies image three inthe sequence was retried once.

The GPS indicates an exact location of the camera during capture foreach image. The azimuth tells us the direction in which the camera ispointed with reference to magnetic or true north. Gravity and deviceorientation provides information on a relative orientation of the camerain image or world co-ordinate systems. The camera level, horizon, etc.allow the system to precisely locate the camera in 3D space for eachimage. Additionally, this provides, in one embodiment, an option to plotthe default look angles for the resulting 3D model based on the initialcamera positions. For example, the first look angle is typicallyestablished with reference to sequence 1 in the capture.

Some other factors that affect how the look angles are establishedinvolve the type of 3D model reconstruction:

if the 3D model caters to roof only geometries, the look angles areestablished at an arbitrary height to the roof of the building, insteadof being at ground level;

if the 3D model is a full building reconstruction, the system mayestablish the look angle starting with a corner, for example, thetypical location of the second image in the sequence;

if the user is interested in visualizing doors in house, the look anglesare established starting with the first camera position at the front ofthe door and plotted in sequence next to different door geometries insequence (or any other desired architecture feature, e.g., windows,dormers, etc.). For example, if the user is interested in visualizingsiding, the look angles may be established starting with side numberone, or camera image 1; or

if the imagery is all captured at a specific altitude, for example froma drone, the look angles may be established at a specific altitude overthe 3D model to provide this perspective by default.

In one embodiment, user or model creation algorithms can manually locatethe look angles after the 3D model is defined by moving the model in 3Dspace and labelling locations within that space. Some reasons for thiscould be that some parts of the 3D model were not fully modeled due tofactors like lack of adequate imagery, known obstructions, etc. Also,for visualizations purposes some angles may not be useful for theviewer, for example the side of the building facing a hill side.

In one embodiment of capture sequencing, arbitrary factors are used tolocate the image capture as it depends on local building constraints.For example, if there is an obstruction from a tree or anotherstructure, the system can direct the user of the camera to repositionthe camera for the next image. In another example, if the camera is tooclose to the building, i.e., the top, bottom, and/or end points are notwithin a frame of the camera, the system can direct the user of thecamera to reposition the camera further back from the same relativeposition. Any repositioning direction is envisioned within the scope ofthe present technical description to assist in establishing look anglesfor a resulting 3D model.

A field of view (extent of the observable view) is set in the next step(503) once the optimal look angle for the selected 3D building model isdefined. The field of view represents the selected 3D building modelincluding its surrounding area. For example, the field of view mayinclude several feet of surroundings in the front of the 3D buildingmodel, the sides of the 3D building model or above the 3D buildingmodel. In one embodiment, the field of view is defined manually byselection of a boundary for a selected 3D building object. In certainembodiments, the 3D building model visualization system defines thefield of view automatically, using computer vision algorithms to accountfor road networks in the vicinity of the selected 3D building model,obstructions from other buildings or structures, relevant faces of thebuilding models, or any other structures or map elements which suggestsan initial optimal field of view. In yet other embodiments, the 3Dbuilding model visualization system provides a default field of viewwhich includes fixed up/down angles (typically 45 degrees) and orleft/right angles.

The first (primary) and second (perpendicular) axes of the selectedbuilding models are calculated by the system (step 504) in the nextstep. In one embodiment, angular thresholds are used to establish groupsof like angles and identify the edges of the structure (e.g., 3Dbuilding model) in order to determine four sides as well as theorientation of a first optimal view. In this exemplary embodiment, afirst edge becomes an initial grouping plane location. As edges areadded to a specific grouping, a weighted edge length is generated thatrepresents a new edge length raised to the power of the influence value.In one embodiment the value of the influence value is 3. In otherembodiments, if the influence value is zero, then a main axis iscalculated as the average of all of the edge orientations with theangular threshold regardless of their length. The larger that this valueis, the more that longer edges and less that shorter edges haveinfluence over the estimate of the axis going through the edges. Aweighted edge plane is generated for each edge that is the distance ofthe new edge from the initial edge plane multiplied by the groupingplane normal minus the initial edge plane multiplied by the weightededge length. The weighted length sum is maintained as the sum of theweighted edge lengths. The sum of the weighted edge planes ismaintained. The grouping plane that the edges are adjusted to ismaintained as the sum of the weighted edge planes divided by theweighted length sum. While this method of establishing groups of likeangles reflects an exemplary embodiment, other methods of recognizinglike angles are envisioned without departing from the scope of thetechnology described herein.

In a next step, a camera position is calculated (step 505). The methodof calculating the camera position is discussed with respect to FIG. 6below. In a final step, an optimal view of the selected 3D buildingmodel is generated (step 506) by a 3D building model visualizationsystem according to the process and is communicated to the electronicdevice for display (step 507).

FIG. 6 illustrates a flowchart of an exemplary embodiment forcalculating a camera position in accordance with the present disclosure.The process involves calculating the camera position as discussed inblock 505 of FIG. 5. In a first step, a projection pyramid 501 as shownin FIG. 4 projecting from the camera is defined (step 601). In oneembodiment, a projection pyramid projecting from the camera point isdefined manually. Yet in other embodiments, the projection pyramidprojecting 401 from the camera point is set automatically by 3D buildingmodel visualization system. An offset angle up and down from forwarddirection vector 402 for the camera are equal, and the offset anglesleft and right of the forward direction vector 402 are equal. The offsetangles are smaller than the defined field of view established in process500 of FIG. 5. In one or more embodiments, by default, the up and downoffset angles from forward direction vector 402 are set to 20 degrees togive a 2.5 degrees vector up and down. The left and right offset anglesare adjusted based on the aspect ratio of the viewer window. In oneembodiment, the corner vectors defining the up-right, down-right,down-left, and up-left edges projecting from camera, as well as sidevectors defining projections along the center-right, center-down,center-left, and center-up are stored by the 3D building modelvisualization system. In another embodiment, the side vectors definingprojections along the center-right, center-down, center-left, andcenter-up are stored by the 3D building model visualization system.

In a next step, lower optimal views and top optimal views are generatedfor each selected 3D building model (step 602). In an exampleembodiment, at least one for the positive direction of each axis and atleast one for the negative direction of each axis (the primary axis andperpendicular axis) providing four lower optimal views and four topoptimal views.

In another step, to define the camera vectors (step 603), the systemdetermines and calculates the forward direction vector as well as rightand up camera vectors. The forward camera direction for lower optimalviews is calculated as being in the same vertical plane of thecalculated axis and using the defined downward pitch angle. The rightvector 403, which will be parallel to the right direction of a display,is calculated as being 90 degrees clockwise to the forward cameradirection vector and horizontal. The up vector 404, which will beparallel to the up direction of the viewer screen, is calculated asbeing rotated 90 degrees up from the forward direction vector and alsoas being in the same vertical plane of the calculated axis.

In a next step (step 604), the vertex in the building model structureselection that is furthest forward along the camera forward directionvector 402 is found and a projection plane (not shown) is defined asbeing normal or perpendicular to the camera forward direction vector andgoing through the vertex that is furthest forward along the cameraforward direction vector (step 605).

In one embodiment, the projection pyramid vectors are rotated (step 606)so that the camera forward direction vector is centered between theup/down and left/right sides and between the vectors that define thepyramid. In another embodiment, the projection pyramid vectors arerotated so that the camera forward direction vector is centered betweenthe up/down and left/right sides that define the pyramid.

In one embodiment, in a next step (step 607), for each vertex in the 3Dbuilding model structure selection, lines are projected from the vertexin the direction of each corner pyramid vector are intersected with theprojection plane. In one embodiment, if the vertex is below the terrain,the vertex location is snapped vertically to the terrain height, andthat position is used instead. In another embodiment, for each vertex inthe building model structure selection, lines are projected from thevertex in the direction of each side pyramid vector are intersected withthe projection plane. In certain embodiments, if the vertex is below theterrain, the vertex location is snapped vertically to the terrainheight, and that position is used as the intersection point instead.

In a next step (step 608), the minimum and maximum distances of theintersection points along the camera up and camera right directionvectors from a reference point on the projection plane are stored alongwith the vertices that the projection points came from. These distancesdefine a bounding box on the projection plane around the vertex lineintersections.

The up and down extreme points are then projected onto a vertical planenormal or perpendicular to the camera right vector while the left andright extreme points are projected onto a plane normal or perpendicularto the camera up vector (blocks 609 and 610).

Convergence points away (back) from the projection plane are thencalculated (step 611) in two stages. In a first stage, a convergencepoint away from the top and bottom edges of the bounding box iscalculated according to the following steps. In a first step, the normalof the top surface of the projection pyramid is calculated as the crossproduct of the camera right vector and the projection pyramid center-updirection vector. In a second step, a convergence plane is defined asthe plane that has a normal the same as the top surface of theprojection pyramid and going through the vertex with highestintersection on the projection plane. In a third step, a convergenceline is defined as the line with the direction the same as theprojection pyramid center-down direction vector and going through thevertex with the lowest intersection on the projection plane. In a finalstep, the intersection between the convergence plane and the convergenceline produces the up/down convergence point.

In a second stage, a convergence point away from the left and rightedges of the bounding box is calculated according to the followingsteps. In a first step, the normal of the right surface of theprojection pyramid is calculated as the cross product of the camera upvector and the projection pyramid center-right direction vector. In asecond step, a convergence plane is defined as the plane that has anormal the same as the right surface of the projection pyramid and goingthrough the vertex with furthest right intersection on the projectionplane. In a third step, a convergence line is defined as the line withthe direction the same as the projection pyramid center-left directionvector and going through the vertex with the furthest left intersectionon the projection plane. In a fourth step, the intersection between theconvergence plane and the convergence line to get the left/rightconvergence point.

The distance of the camera from the projection plane is found next (step612) by measuring the distance between the up/down convergence point andthe projection plane and the distance between the left/right convergencepoint and the projection plane. The camera is positioned back from thecenter of the projection plane bounding box at the largest convergencepoint distance from the projection plane.

In one or more embodiments, the camera position is adjusted for bettercentering (step 613). If the up/down convergence distance from theprojection plane is shorter than the left/right convergence distance,the calculations can be iterated with smaller up/down offset angles fromthe forward vector until the up/down convergence distance substantiallymatches the left/right convergence distance. If the left/rightconvergence distance from the projection plane is shorter than theup/down convergence distance, the calculations can be iterated withsmaller left/right offset angles from the forward vector until theleft/right convergence distance substantially matches the up/downconvergence distance.

FIGS. 7A and 7B illustrate examples of optimal views for 3D buildingmodels in the 3D building model visualization system in accordance withthe present disclosure. FIG. 7A illustrates a screen-shot of multiplebuilding models on a street within the disclosed 3D building modelvisualization system. In one embodiment, one or more building models 701are selected, for example, by typing in the exact address of a buildingmodel on the map. In one embodiment the address is entered in the 3Dbuilding model visualization system in the form of a mailing address,GPS/geographical coordinates of the building, or as a point of interest.In other embodiments, the 3D building model is selected by clicking onit with a computer mouse, joystick or the like. The typing of theaddress may be done using known computer keyboard devices, voice or handwaving commands or any other known and available technology of typingdata into the system. In other embodiments, the building model isselected within the interface by touching or swiping on a touch-screendisplay or a capacitive screen display (e.g., using a finger, stylus orthe like). In one embodiment, upon selection of a 3D building model(e.g., structure or object), the 3D building model visualization systemcalculates or retrieves a pre-calculated primary optimum view 702 of thebuilding model as shown in FIG. 7B.

FIG. 8 illustrates a general schematic of an embodiment in accordancewith the present disclosure. In one embodiment, the 3D building modelvisualization system displays the optimal views for a selected 3Dbuilding model. Instead of a single view (e.g., the primary optimalview), the 3D building model visualization system displays optimum views801 through 805 of the building model's facades. Each of the displayedviews are optimal views (e.g., views corresponding to downwards lookangles as depicted in FIGS. 3A and 3B).

FIG. 9 illustrates an embodiment for displaying and navigating a 3Dbuilding model visualization system in accordance with the presentdisclosure. In a first embodiment, selection of a building model 900from the 3D building model visualization system automatically generatesand displays optimal views of the building model as fixed perspectiveviews (e.g., front, left, right, top, rear and bottom). While a mousecursor 901 was used in this figure to show selection of a building modelin the 3D building model visualization system, any other known selectiondevices and methods are also contemplated. Selection of a building modelby joy-stick, game controller, touching a touch sensitive screen, typingan address or geographic coordinates, automatic selection using alocation, or by using voice commands, eye gazing or hand waving commandsproduce the same results. Interface navigation controls 902 (e.g., up,left, right and down arrows) allow for visualization of a building modeland/or navigate in the 3D building model visualization system throughfixed, optimal perspectives views 903 through 907 of the building model900, simplifying the traditional “six degrees of freedom” fine motoradjustments.

Throughout the specification, drawings and claims various terminology isused to describe the various embodiments. As may be used herein, theterms “substantially” and “approximately” provides an industry-acceptedtolerance for its corresponding term and/or relativity between items.Such an industry-accepted tolerance ranges from less than one percent tofifty percent and corresponds to, but is not limited to, measurements,angles, positions and geo-locations. Such relativity between itemsranges from a difference of a few percent to magnitude differences.

The technology as described herein may have also been described, atleast in part, in terms of one or more embodiments. An embodiment of thetechnology as described herein is used herein to illustrate an aspectthereof, a feature thereof, a concept thereof, and/or an examplethereof. A physical embodiment of an apparatus, an article ofmanufacture, a machine, and/or of a process that embodies the technologydescribed herein may include one or more of the aspects, features,concepts, examples, etc. described with reference to one or more of theembodiments discussed herein. Further, from figure to figure, theembodiments may incorporate the same or similarly named functions,steps, modules, etc. that may use the same or different referencenumbers and, as such, the functions, steps, modules, etc. may be thesame or similar functions, steps, modules, etc. or different ones.

While particular combinations of various functions and features of thetechnology as described herein have been expressly described herein,other combinations of these features and functions are likewisepossible. For example, the steps may be completed in varied sequences tocomplete the textured facades. The technology as described herein is notlimited by the particular examples disclosed herein and expresslyincorporates these other combinations.

What is claimed is:
 1. A method of visualizing a multi-dimensionalbuilding model on a computer display, the method comprises: retrieving amulti-dimensional building model from computer memory; defining a lookangle for the multi-dimensional building model, the defining a lookangle linking a real camera to virtual camera positioning based oncontextual information obtained during image capture of themulti-dimensional building model; defining a field of view for themulti-dimensional building model by defining up, down, left and rightangles which define an extent of the multi-dimensional building model tobe viewed on the computer display; calculating a first main axis and asecond main axis for the retrieved multi-dimensional building model todefine an orientation of the multi-dimensional building model;calculating an optimal camera position for the retrievedmulti-dimensional building model based on the defined look angle, thedefined field of view and the orientation, the optimal camera positiondefining an optimal view of the retrieved multi-dimensional buildingmodel; and displaying the optimal view on the computer display.
 2. Themethod of claim 1, wherein the contextual information about the imagecapture includes, but is not limited to: GPS Latitude Ref; GPS LongitudeRef; GPS Altitude; GPS Dilution of Precision; GPS Img (image) DirectionRef (reference); GPS Img (image) Direction; metadata for gravity;metadata for camera orientation; or or an image capture sequence number.3. The method of claim 2, wherein the image capture sequence numberdescribes and associates each image with a virtual location.
 4. Themethod of claim 2, wherein the image capture sequence includes imagecapture starting at a front door, with counter-clockwise capture tosequentially capture images of one or more of: front right corner, rightside, back left corner, back side, back right corner, side, front leftcorner in an image capture sequence with a name associated with imagesin the image capture sequence.
 5. The method of claim 2, wherein, if anyimage of an image capture sequence is recaptured, the image capturesequence number is preserved.
 6. The method of claim 1, wherein thecontextual information is included with each image via an EXIF(exchangeable image file) format tag or separately via a metadata file.7. The method of claim 1 further comprises plotting default look anglesfor multi-dimensional building models based on an initial position ofthe real camera.
 8. The method of claim 1 further comprises determininga first look angle based on a first image in an image capture sequence.9. The method of claim 1 further comprises determining the look anglebased on the contextual information including any of: an arbitraryheight relative to a roof of a building; a corner of a building;architectural features plotted in sequence; a specific altitude abovethe multi-dimensional building model; labeled locations; or arbitraryfactors used to locate the image capture as it depends on local buildingconstraints wherein the optimal view includes an unobstructed view. 10.The method of claim 1, wherein the retrieved multi-dimensional buildingmodel includes pre-stored, previously calculated ones of centered, atleast partially unobstructed, optimal views for each façade.
 11. Asystem for visualizing a multi-dimensional building model, the systemcomprising: a view processor for processing an optimal view of amulti-dimensional building model, the processor configured to: retrievethe multi-dimensional building model from computer memory; define a lookangle for the multi-dimensional building model, the defining a lookangle including linking a real camera to virtual camera positioningbased on contextual information obtained during image capture of themulti-dimensional building model; define a field of view for themulti-dimensional building model by defining up, down, left and rightangles which define an extent of the multi-dimensional building model tobe viewed on a computer display; calculate a first main axis and asecond main axis for the retrieved multi-dimensional building model todefine an orientation of the multi-dimensional building model; calculatean optimal camera position for the retrieved multi-dimensional buildingmodel based on the defined look angle, the defined field of view and theorientation, the optimal camera position defining an optimal view of theretrieved multi-dimensional building model; and communicate the optimalview to a remote electronic device to be displayed on the computerdisplay of the remote electronic device.
 12. The system of claim 11,wherein the contextual information about the image capture includes, butis not limited to: GPS Latitude Ref; GPS Longitude Ref; GPS Altitude;GPS Dilution of Precision; GPS Img (image) Direction Ref (reference);GPS Img (image) Direction; metadata for gravity; metadata for cameraorientation; or or image capture sequence number.
 13. The system ofclaim 12, wherein the image capture sequence number describes andassociates each image with a virtual location.
 14. The system of claim12, wherein the image capture sequence includes image capture startingat a front door, with counter-clockwise capture to sequentially captureimages of one or more of: front right corner, right side, back leftcorner, back side, back right corner, side, front left corner in animage capture sequence with a name associated with images in the imagecapture sequence.
 15. The system of claim 14, wherein, if any image ofan image capture sequence is recaptured, the image capture sequencenumber is preserved.
 16. The system of claim 11, wherein the processoris further configured to plot default look angles for amulti-dimensional building model based on an initial real cameraposition.
 17. The system of claim 11, wherein the processor is furtherconfigured to determine a first look angle based on a first image in animage capture sequence.
 18. The system of claim 11, wherein theprocessor is further configured to determine the look angle based onmetadata associated with the contextual information including any of: anarbitrary height relative to a roof of a building; a corner of abuilding; architectural features plotted in sequence; a specificaltitude above the multi-dimensional building model; labeled locations;or arbitrary factors used to locate the image capture as it depends onlocal building constraints wherein the optimal view includes anunobstructed view.
 19. A method of visualizing a multi-dimensionalbuilding model on a mobile electronic device display, the methodcomprises: selecting a multi-dimensional building model based on alocation of the multi-dimensional building model; wirelessly retrieving,from remote computer memory, a previously stored optimal view of theselected multi-dimensional building model, the optimal view based on atleast linking a real camera to virtual camera positioning based oncontextual information obtained during image capture of themulti-dimensional building model, and wherein for the selectedmulti-dimensional building model, the optimal view is based on a lookangle, a field of view and an orientation of the selectedmulti-dimensional building model, wherein the optimal view includes acentered at least partially unobstructed view of the selectedmulti-dimensional building model; and displaying the optimal view on themobile electronic device display.
 20. The method of claim 19, whereinthe contextual information about the image capture includes, but is notlimited to: GPS Latitude Ref; GPS Longitude Ref; GPS Altitude; GPSDilution of Precision; GPS Img (image) Direction Ref (reference); GPSImg (image) Direction; metadata for gravity; metadata for cameraorientation; or or image capture sequence number.